[RFC 0133] Git hashing and Git-hashing-based remote stores #133
Conversation
Ericson2314
changed the title
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/obsidian-systems-is-excited-to-bring-ipfs-support-to-nix/7375/43 |
Ericson2314
changed the title
Thanks! Co-authored-by: Kevin Cox <kevincox@kevincox.ca>
|
I would like to nominate @kevincox based on his feedback already (yay!) and his participation in the previous RFC #122 meetings. I also suspect we'll need one or both of @edolstra and @thufschmitt for changes to Nix like this, so I nominate them both too. I also suspect @fricklerhandwerk and @tomberek might be interested also, time permitting. I would not mind extra shepherds for this one! |
|
I love the possibility of distributed content-addressed caches. I accept the nomination. |
edolstra
added
status: new
status: open for nominations
|
This RFC is open for shepherd nominations! |
|
@Ericson2314 Sorry, I can't commit any time to this. I like the idea a lot, but would have to go into more depth to be of any help, and this would be too much at the moment. |
|
@fricklerhandwerk No problem at all :). I figured might be the case, but wanted to extend the nomination anyways based on our prior conversation. |
|
I accept the shepherd nomination. We need at least one more shepherd. Any takers? |
|
On behalf of the shepherd committee, I declare this RFC to be in the Final Comment Period! |
kevincox
added
status: FCP
Co-authored-by: Kevin Cox <kevincox@kevincox.ca>
They are now in an `ipfs-2` branch in this repo.
That can go in a separate blog post.
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfc-0133-fcp-git-hashing-and-git-hashing-based-remote-stores/29749/1 |
|
different versions of IPFS have different default chunking parameters so ipfs add gives different CIDs. you have two options:
guess what, There is defaults in the reference implementation of ipfs (balanced dag, 256KiB dag, protobuf wrapping) but pinning services don't follow them necessarily. Additional info: Here is a google 'version' of IPFS, note the segmentation of the hash in the URI format: |
Thank you both!! Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Co-authored-by: Ryan Lahfa <masterancpp@gmail.com>
The other stuff is already in flight, we don't need to talk about it so much here. Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
infinisil
added
status: accepted
and removed
status: FCP
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/nixcon-governance-workshop/32705/9 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
Excited to report we now have the first part implemented: the For how to use this, one can consult https://github.com/NixOS/nix/blob/master/tests/functional/git-hashing/simple.sh In particular, nix store add --mode git --hash-algo sha1 <file>should work! |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/obsidian-systems-is-excited-to-bring-ipfs-support-to-nix/7375/65 |
* ipfs: Copy Template * ipfs: Start drafting * ipfs: Finish draft * ipfs: Expand discussion of managing complexity * ipfs: Fix typos Thanks! * ipfs: Fix more typos Thanks! * ipfs: FInish motivation on source distribution and archival * ipfs: Rename now that we have number * Apply suggestions from code review Thanks! Co-authored-by: Kevin Cox <kevincox@kevincox.ca> * Fix typos Thanks! Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com> * 133: Add shepherd team! Co-authored-by: Eelco Dolstra <edolstra@gmail.com> * 133: Fix shepherds list mjoseph -> amjoseph * 133: Move non-`git` steps to future work * 133: Move one more section out of future work * 133: Move IPFS-specific motivation to future work too * 133: Rename feature in light of changes * 133: Rename RFC in light of changes * 133: Discuss the downside of git's file system model being different * Split future work, clean up Nix-agnostic stores section * Fix numerious typos Thanks, all of you! Co-authored-by: Kevin Cox <kevincox@kevincox.ca> Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com> Co-authored-by: Linus Heckemann <git@sphalerite.org> * Add RFC open PR date * Be clearer about not supporting references to start * Update rfcs/0133-git-hashing.md Co-authored-by: Kevin Cox <kevincox@kevincox.ca> * Rip out both RFC-scal Future Work sections They are now in an `ipfs-2` branch in this repo. * Remove "Build adoption through seamless interop" That can go in a separate blog post. * Apply suggestions from code review Thank you both!! Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Co-authored-by: Ryan Lahfa <masterancpp@gmail.com> * Slim down the layering section The other stuff is already in flight, we don't need to talk about it so much here. Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> --------- Co-authored-by: Kevin Cox <kevincox@kevincox.ca> Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com> Co-authored-by: Eelco Dolstra <edolstra@gmail.com> Co-authored-by: Linus Heckemann <git@sphalerite.org> Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io> Co-authored-by: Ryan Lahfa <masterancpp@gmail.com>
Teach Nix Git hashing for store objects along-side flat and NAR hashing.
Also teach it to use remote stores that don't know about Nix concepts but do know about git hashing as substituters.
This follows the work done and described in https://github.com/obsidiansystems/ipfs-nix-guide/. The idea is that IPFS can act as one such store, but Nix need not know about IPFS per se, since they both "speak" Git hashing. Also, software heritage can act as another.
Rendered Merged
Tracking issue / Milestone